Time-Series Regression and Generalized Least Squares in R An Appendix to An R Companion to Applied Regression, Second Edition

Generalized least-squares (GLS ) regression extends ordinary least-squares (OLS) estimation of the normal linear model by providing for possibly unequal error variances and for correlations between different errors. A common application of GLS estimation is to time-series regression, in which it is generally implausible to assume that errors are independent. This appendix to Fox and Weisberg (2011) briefly reviews GLS estimation and demonstrates its application to time-series data using the gls function in the nlme package, which is part of the standard R distribution. 1 Generalized Least Squares In the standard linear model (for example, in Chapter 4 of the text), y = X + " where y is the n × 1 response vector; X is an n × k + 1 model matrix; is a k + 1 × 1 vector of regression coefficients to estimate; and " is an n×1 vector of errors. Assuming that " ∼ Nn(0, In) leads to the familiar ordinary-least-squares (OLS ) estimator of , bOLS = (X ′X)−1X′y with covariance matrix Var(bOLS) = 2(X′X)−1 Let us, however, assume more generally that " ∼ Nn(0,Σ), where the error covariance matrix Σ is symmetric and positive-definite. Different diagonal entries in Σ correspond to non-constant error variances, while nonzero off-diagonal entries correspond to correlated errors. Suppose, for the time-being, that Σ is known. Then, the log-likelihood for the model is loge L( ) = − n 2 loge 2 − 1 2 loge(det Σ)− 1 2(y −X ) ′Σ−1(y −X ) which is maximimized by the generalized-least-squares (GLS ) estimator of , bGLS = (X ′Σ−1X)−1X′Σ−1y with covariance matrix Var(bGLS) = (X ′Σ−1X)−1